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1 . A device for creating a three-dimensional profile of an object or scene 
being imaged, the device comprising: 

a light source for illuminating said object or scene with a light pattern, wherein 
said light pattern has a characteristic that varies across said object or scene and that 
includes two or more sub-patterns; 

an image sensor for imaging said object or scene as illuminated with said light 
pattern; and 

a processor for calculating a distance to a point on said object or in said scene 
based on a baseline distance between said light source and said camera, an angle between 
said camera and said baseline, and an angle at which light striking the point is emitted by 
said light source as determined from the characteristic of the light striking the point. 

2. The device of claim 1, wherein the varying characteristic is a spatially 
varying wavelength, and wherein two or more sub-patterns are two or more spectra of 
light in the visible light spectrum such that a given wavelength corresponds to more than 
one projection angle. 

3 . The device of claim 2, wherein the processor determines the projection 
angle for the given wavelength via adaptive initial point calculation, wherein the 
processor restricts a search space for color matching in one of said sub-patterns. 

4. The device of claim 1 , wherein the two or more sub-patterns are two or 
more spectra of light in the infrared light spectrum. 

5. The device of claim 1, wherein the two or more sub-patterns are two or 
more spectra of light in the ultraviolet light spectrum. 
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6. The device of claim 1, wherein the varying characteristic is a spatially 
varying intensity, and wherein two or more sub-patterns are three light sub-patterns 
corresponding to red, green and blue components of the light pattern. 

7. The device of claim 6, wherein said three light sub-patterns have varying 
intensities, each sub-pattern having a high point and a low point. 

8. The device of claim 7, wherein the high points and low points for the three 
sub-patterns are distributed over a spatial period of the light pattern. 

9. The device of claim 1 , wherein the varying characteristic is a spatially 
varying wavelength, and wherein the device further comprises an optical filter coupled to 
the light source to generate said light pattern. 

10. The device of claim 9, wherein said optical filter is a linear variable 
wavelength filter. 

1 1 . The device of claim 1 , wherein the light pattern is pre-distorted based on a 
pre-calibration characteristic of the image sensor. 

12. The device of claim 1 , further comprising a second image sensor, wherein 
said image sensor and said second image sensor together form a stereo pair. 

13. The device of claim 1, wherein the varying characteristic is a spatially 
varying intensity and wherein the image sensor is monochromic and the light source 
generates the light pattern by sequentially emitting a plurality of light sub-patterns. 

14. The device of claim 13, wherein the plurality of light sub-patterns is three 
light sub-patterns corresponding to red, green and blue components of the light pattern. 
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DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 



[0037] Figures 6a and 6b illustrate one manner in which the light projection pattern can 
be modified to improve resolution and measurement accuracy. Figures 6a and 6b assume 
that the light pattern is a single rainbow projection pattern. As explained above, the light 
pattern contains red, green and blue (RGB) components. In one embodiment of the 
invention, the intensities of the RGB components in the light pattern are modulated so 
that the intensity of each R, G, B component varies between high and low points, i.e. 
from a peak to a valley, and forms multiple cycles of pattern variation. The locations of 
the peak-valley points for the R, G, and B sub-patterns are spread over the spatial period 
of the light pattern, as shown in Figure 6b. Notice that the distribution of R, G, and B 
does not need to be evenly spaced across the spatial period. 

[0038] Because this embodiment uses a single rainbow projection pattern as the light 
pattern, the one-to-one correspondence between the wavelength of a light sheet in the 
light pattern and its projection angle still applies in this embodiment. 
[0039] Referring now to Figures 7a through 7c, the light pattern from the light projector 
does not have to be limited to a single spectrum, where a one-to-one correspondence 
between a given wavelength and a given projection angle applies. Instead, the light 
pattern may include multiple cycles of spectra, such as multiple visible light spectra, such 
that a given wavelength from the light projector could be generated and transmitted in 
several projection directions, i.e. at several projection angles. This will be referred to as a 
"multiple rainbow projection" pattern, even though the spectra is not limited to the visible 
rainbow light spectrum. Because a given wavelength will no longer have a one-to-one 
correspondence with a projection angle, the look-up table will contain two or more 
projection angles for each wavelength (a one-to-many relationship). As a result, the 
multiple rainbow projection system requires an additional procedure to select the correct 
projection angle for a given wavelength. 

[0040] Figures 8 and 9 is a graph and flowchart, respectively, outlining the procedure for 
conducting color matching for a one-to-many lookup table that contains multiple 
projection angles for a given wavelength. To avoid any mismatching from an incorrect 
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projection angle selection, the method shown in Figure 9 can be used to reduce any 
ambiguities. 

[0041] First, adaptive control can be used to determine the initial condition in the search. 
More particularly, an adaptive initial point calculation mechanism may be used to ensure 
that the initial point at which the iterative search method begins is at least somewhat 
close to the correct result, particularly because the final result of iterative processes tends 
to depend on the selected initial condition. By limiting the search range, as shown in 
Figure 8, within one spectrum in the multiple-spectrum light pattern, the likelihood of a 
correct color match is greatly increased. 

[0042] One method of increasing the chance for having a good initial point in a search is 
to use an adaptive initial point calculation mechanism. The initial point of the search is 
determined by a correct match point in the search of a neighboring pixel The underlying 
assumption here is that most portions of the surface of a physical object being imaged are 
continuous. Thus the projected, i.e., reflected, color is similar for neighboring portions of 
the object, and therefore the initial projection angle value should also be very similar. 
[0043] As can be seen in Figure 9, the process starts by determining origin point 
coordinates (xo, yo, z 0 ) at step 900. This point can be, for example, the upper right-hand 
pixel in the image. To compute the correct value for the desired point (x'j, y'j, z',) at step 
902, the inventive system uses color matching to obtain projection angle candidates at 
step 904. For example, if the object is being illuminated with two identical rainbow 
spectra are arranged horizontally, i.e., right and left, with respect to each other, it can be 
assumed that the light resulting in the upper right-hand pixel of the image taken results 
from light in the right-hand spectrum as reflected by the object. Therefore, to identify the 
angle at which that light was originally emitted, 9, the wavelength of that light is matched 
to the angle at which that wavelength is being emitted in the right-hand, as opposed to the 
left-hand, rainbow spectrum. Once this angle is identified, the value for (x'j, y'i, z'j) 
obtained at step 906 is compared with previously obtained neighborhood point (Xj_i, y^i, 
Zj-i) to determine the correct value for (xi, y;, Zj) at step 908. If the iteration process is 
determined to be complete at step 910, then the process ends. Otherwise, another 
iteration is conducted to refine the determination of the value of (x;, yj, Zi). 
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[0044] Note that although the look-up table for a multiple rainbow projection system has 
multiple projection angles associated with any given wavelength when the light pattern is 
viewed as a whole, the system restricts the search space in the color matching operation 
within a smaller range (e.g., within one sub-pattern) so that, as a practical matter, there is 
a one-to-one correspondence between the wavelength and the projection angle when the 
search is restricted to within the sub-pattern. 

[0045] Referring to Figures 10a and 10b, the actual projection pattern appearing on the 
image received by the camera is the convolution of the projection pattern produced by the 
light source and the sensitivity characteristics of the image sensor. As shown in the 
Figure 10a, if an image sensor has a nonlinear sensitivity, a linear projection pattern 
provided by the light source may be distorted into a nonlinear variation pattern in the 
received image 

[0046] To compensate for any nonlinear characteristics of the camera 102 or other image 
sensor used in the system, it may be desirable to pre-calibrate the characteristics of the 
image sensor 102, then apply its inverse function to the desired projection pattern design, 
resulting in a pre-distorted projection pattern, as shown in Figure 10b. The pre-distortion 
projection pattern compensates for any non-linearities in the camera or image sensor so 
that the final pattern on the image captured by the image sensor is ideal and distortion- 
free. 

[0047] Although all of the above examples assume that there is only one camera or image 
sensor used to capture images of the illuminated object, multiple cameras can also be 
used to capture multiple rainbow projection pattern data. Referring to Figure 1 1, the light 
projector, marked CD, generates the multiple rainbow projection patterns, marked ©, to 
lighten the scene containing 3D objects of interest, marked ©. The reflected light is 
detected by a pair of color CCD cameras, marked CD and ©. The images, which contain 
both intensity and color components, will be grabbed by a real time frame grabber board, 
marked ©, into a host computer, marked ©, to perform pixel-to-pixel color matching and 
triangulation calculation, based on the similarity of color patterns and intensity variation 
in the neighborhood of pixels in stereo image pairs. 
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[0048] The color spectrum of pixels on the captured images is determined by the 
proportion of Red, Green and Blue components (RGB) associated with the individual 
pixel. As a result, pixel-to-pixel registration can be easily performed based on matching 
the color components of counterparts. Because the length of the baseline B between the 
two cameras is known, and the geometrical and optical parameters of two cameras can be 
obtained from a prior calibration procedure, the locations of a pair of corresponding 
pixels in a stereo image pair provide sufficient information to determine viewing angles 
otl and cc2. The range values, R, associated with each pixel between a camera and 
surface points can be obtained using a straightforward triangulation: 



Thus, a full frame of a three-dimensional image can be obtained from a single snap shot, 
and a stream of three-dimensional images can be generated at the camera's frame rate 
(e.g., 30 frames per second or higher). 

[0049] By using multiple cameras in conjunction with the multiple rainbow projection 
system, three-dimensional measurement accuracy is greatly improved, particularly when 
measuring objects and scenes that do not have many distinguishing surface features. 
More particularly, the stereo matching and triangulation methods used in this 
embodiment reduces the dependence of distinguishing features, such as edges and 
corners, on an object's surface in conducting the measurement. 
[0050] Referring now to Figures 12a and 12b, the inventive system is not limited to 
systems using a color camera or color images, but can also be implemented using a 
monochromic camera and monochromic projection patterns. More particularly, a 
multiple rainbow projection pattern can be represented as a combination of three 
individual bands of color, namely red, green, and blue (or other colors, if a different color 
space is defined). Each band of color components (R, G, B) can be projected sequentially 
so that a monochromic camera can collect the reflected images for each color component 
upon each projection. These images are then combined to form a composite "color" 
image, which is equivalent to the color image produced by a color camera under the 
multiple rainbow projection, as illustrated in Figure 12a. Figure 12b outlines the method 
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corresponding to the sequential images illustrated in Figure 12a. The resulting composite 
image can be used like any color image and can be captured by systems employing one 
camera or multiple cameras. 

[0051] Using a monochromic camera to obtain the color image generates high-resolution 
images using lower-cost, less complex equipment. However, because multiple images 
need to be obtained of any given scene to generate a full-frame three-dimensional image, 
this embodiment is most suitable for acquiring images of static or slowly moving objects. 
[0052] Note that the above examples assumed that an optical filter is used to produce the 
illumination pattern, such as a rainbow pattern, on the object or scene being imaged. In 
this type of system the light from the light source is projected through the optical filter to 
generate the light pattern. The quality of the light pattern, however, depends greatly on 
the quality of the optical filter, which can be expensive and difficult to produce. 
[0053] Figures 13 through 16a and 16b illustrate an alternative structure that can create 
multiple-periods of a light pattern having gradually changing intensities. Instead of using 
an optical filter, a planar member 1000 with slots 1002 cut through the member to form a 
slotted plate 1000, as shown in Figures 13 and 14, may be used. The slotted plate 1000 
can be made of any non-transparent material and have any physical dimensions and any 
thickness. In a preferred embodiment, the width of each slot 1 002 is made to be the same 
as the width of the material between the slots. Further, as illustrated in Figure 14, the 
cross-sectional shape of the slots themselves can have any desired shape and do not need 
to conform to any specific dimensions. The slotted plate 1002 can be placed in a similar 
position as the optical filter, in front of the light source so that the light from the source is 
projected toward the object being imaged through the slotted plate. As a result, a light 
projector using the slotted plate 1000 is able to generate the desired spatially- varying 
illumination pattern on objects in a scene with varying light intensity crossing the scene. 
[0054] To generate the illumination pattern, assume that the illumination intensity on the 
surface of an object in a scene can be expressed as a function of the coordinate and the 
contribution of each point light source. The expression for the intensity then becomes: 



Intensity = 
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